High-accuracy model recognition method of mobile device based on weighted feature similarity

Accurately model recognition of mobile device is of great significance for identifying copycat device and protecting intellectual property rights. Although existing methods have realized high-accuracy recognition about device’s category and brand, the accuracy of model recognition still needs to be improved. For that, we propose Recognizer, a high-accuracy model recognition method of mobile device based on weighted feature similarity. We extract 20 features from the network traffic and physical attributes of device, and design feature similarity metric rules, and calculate inter-device similarity further. In addition, we propose feature importance evaluation strategies to assess the role of feature in recognition and determine the weight of each feature. Finally, based on all or part of 20 features, the similarity between the target device and known devices is calculated to recognize the brand and model of target device. Based on 587 models of mobile devices of 17 widely used brands such as Apple and Samsung, we carry out device recognition experiments. The results show that Recognizer can identify the device’s brand and model than existing methods more effectively. In average, the model recognition accuracy of Recognizer is 99.08% (+ 9.25%↑) when using 20 features and 92.08% (+ 29.26%↑) when using 13 features.


Related work
Currently, there are three main research directions for device recognition: traffic-based device recognition methods, network-search-based device recognition methods, and physical-information-based device recognition methods.
In traffic-based device recognition methods, the authors mine and analyze the attribute information (such as ports, protocols, banners, etc.) in active measurement traffic, and the behavior features (such as packet length, sending interval, and statistical feature of data flow) in passive monitor traffic. After that, the authors build a device fingerprint database or train device-recognition classifiers to realize the discrimination of the device type. In Nmap 8 , the device's network ports are extracted from the active measurement traffic, and the device fingerprint is calculated by those port results. Based on these device fingerprints, Nmap tool identifies the service type and operating system of device. After that, Durumeric et al. 9 develop Zmap, which greatly accelerate the device information collection before recognition. So, Zmap improve the recognition efficiency of network devices. In paper [10][11][12][13][14] , the authors extract the IP, port, flag bits, and others in the header of the TCP packet as features, and use machine learning algorithms to train the recognition classifiers, thereby realizing the discrimination of the device type. Those methods in papers [15][16][17][18] extract features from the protocol data of each layer from passive monitor traffic to form device fingerprints, and identify the type of target device by matching fingerprint. With slight differences in the previously mentioned methods, those methods in paper [19][20][21] extract features (such as packet time, length, port, DNS protocol, etc.) from the network layer data and application layer data in traffic, and use a variety of machine learning algorithms to build a phased recognition classifier to identify device. Cheng et al. 22 recognize the device according to the difference between the file headers of devices in active measurement traffic. To improve the security in training model, He et al. 23 build a recognition model based on federal learning. To improve the usability of the recognition model, Jiao et al. 24 propose a multi-level identification framework to decrease the updating frequency of recognition model when training data is updated. In addition, the methods in 25,26 do not need to extract features from traffic and eliminates the influence of features extraction behavior during the recognition process. The traffic-based device recognition method can easily obtain the measurement data, and can identify device types and brands in batches in normal network environment. In 27 , the authors summarize the traffic-based device recognition methods. In real life, the systems, built based on this type of methods, such as Shodan 28 , ZoomEye 29 , Censys 30 , and Quake 31 have been widely used. However, because same categories of devices with one brand often use same protocol to transmit data, the difference between these devices in traffic is not obvious. This kind of methods is difficult to effectively recognize fine-grained model of device.
In network-search-based device recognition methods, the authors use the Internet crawlers to acquire device information from Internet resources such as URL (Uniform Resource Locator) strings and Web pages, so as to construct a device database for device recognition. Li et al. 32 implement a device recognition algorithm based on the GUI (Graphical User Interface) information in the web pages of camera devices, and found about 1.6 million camera devices. Zou et al. 33 establish an IoT device recognition framework. In this framework, Zou  www.nature.com/scientificreports/ built a device database based on the devices' attributes in the IoT device protocol slogan, and then realized the hierarchical recognition of device. Agarwal et al. 34 develop a tool named WID to recognize the device from the source code of webpages and subpages of devices. ARE 35 could search for special slogans in device webpage, and obtained the device description information from the device annotation to identify device. Those methods can recognize the model of device without building a device fingerprint database or training machine learning classifier. However, because the reliability of Web resources is difficult to evaluate and the webpages' structure of search results is diverse, extracting reliable device information from webpages is complex. This kind of methods are not easy to implement, and the recognition accuracy of those methods is limited in practical work.
In physical-attributes-based device recognition methods, the authors recognize the type of device based on the difference in the physical characteristics. Guo et al. 36 analyze the structure characteristics of device physical addresses and recognize the type of device based on the device's MAC (Media Access Control) addresses. The methods in paper [37][38][39][40][41][42][43] use the time offset characteristics of "the clock of each device is unique, and the deviation still exists after being calibrated by the NTP (Network Time Protocol)" to recognize IoT devices. Radhakrishnan 44 found that the device hardware clock deviation would lead to differences in network behavior. So, Radhakrishnan designs a fingerprint generation algorithm using neural network, named GTID, to identify device types. The device recognition methods based on physical attributes can identify the model of device. Especially, the recognition methods based on the clock offset characteristic, greatly improve the recognition accuracy. However, due to the interference in network, precise time offset of device is difficult to obtain, causing low recognition accuracy of these methods in the actual network. At the same time, measuring the device's clock offset is not easy, which also limits the widely application of these methods.
In this paper, we extract traffic attributes such as GPU model, resolution and operating system, as well as the physical attributes such as device size and screen size, and propose Recognizer, a high accuracy model recognition method of mobile device based on weighted feature similarity. Recognizer extracts the common attributes of all mobile devices as features, and formulates feature similarity calculation rules according to feature expression, so as to measure the similarity between different devices. At the same time, we design the features importance evaluation strategies to assess the role of each feature in brand recognition (we call this strategy "RFBR") and model recognition (we call this strategy "RFMR"). In RFBR and RFMR, the weight of features will be determined. When the target device recognition is performed, brand recognition and model recognition are performed in sequence, so as to obtain the model of target device.

Our method: Recognizer
In this section, we will introduce the principles and steps of the Recognizer in detail.
Symbol description. f: feature. There are three kinds of feature: extracted feature f e , brand feature f b and model feature f m . Among them, f e is the feature extracted from the public attributes of the device, f b is the brand feature selected from the extracted features using the RFBR algorithm, and f m is the model feature selected from the extracted features using the RFMR algorithm. The general representations of the ith extracted feature, brand feature, and model feature are f ( * ,i) , . . . ) and The similarity functions of the kth extracted feature, brand feature and model feature of device D i and D j are denoted as www.nature.com/scientificreports/ we propose RFBR and RBMR strategies to assess the role of each feature in brand recognition and model recognition for feature selection and weight determination. Finally, Recognizer uses the target's features to identify the brand and model. The framework of Recognizer is shown in Fig. 1. There are 7 steps in Recognizer, as follows: Step 1 Group devices. All devices in the knowledge set are grouped by device brand. In each group, all devices' brands are same.
Step 2 Extract feature. In each group, we extract the common attributes of all devices as brand attributes. If all devices in all groups own one attribute, this attribute will be as a feature.
Step 3 Calculate similarity between two features. According to the form of extracted feature, we divide the extracted features into numerical features and string features. For each feature form, we build the feature similarity calculation strategy.
Step 4 Select brand feature. Based on the effect of each feature on the similarity between same-brand devices and the similarity between devices with different brands, we propose RFBR strategy to quantify the importance of each feature in brand recognition, and the importance value is expressed as χ rqb . Those features, whose χ rqb is greater than 0, will be selected as brand features. And the value of χ rqb is as the weight of brand feature.
Step 5 Select model feature. Because one model only corresponds to one mobile device, there is no similarity between devices with same model. So, it is unreasonable to use RFBR strategy for model feature selection.
According to the effect of feature on same-brand devices and the difference of effect on all brands, we propose RFMR strategy to quantify the importance of each feature in model recognition, and the importance value is expressed as χ rqm . Those features, whose χ rqm is greater than 0, are selected as model feature. And χ rqm is the weight of feature.
Step 6 Normalize weights. All weights of brand features obtained in Step 4 and all weights of model features obtained in Step 5 are normalized respectively.
Step 7 Recognize target's model. We obtain brand features and model features from target mobile device. After recognizing the brand of target according to brand features and brand features' weights, the model features and model features' weights are used to identify the model of target.
Key steps of Recognizer. Among all steps of Recognizer, Step 3, 4, 5, 7 are key steps. These key steps are described in detail as follows.
(1) Calculate similarity between two features. www.nature.com/scientificreports/ We divide the extracted features into numerical features and string features. For a feature, if the feature value is a numeric value obtained by measurement tool and there is an inevitable measurable error due to the precision limitation of the measurement tool, the feature is a numeric feature (e.g., length); otherwise, the feature is a string feature (e.g., operating system).
We measure the similarity between two numerical features based on the difference value between two values, while the similarity between two string features is determined based on the inclusion relationship between two strings. Certainly, although a number can be considered as a string, it is not reasonable to calculate the similarity between two numerical features based on the inclusion relationship. For example, for two numeric features f 1 (value is 1000) and f 2 (value is 999), if f 1 and f 2 are regarded as string features, the similarity value between f 1 and f 2 is 0. Obviously, it is unreasonable. Therefore, for two types of features, we design two strategies to calculate the similarity between features, respectively, as follows.
(a) Numerical feature similarity strategy For numerical features, the smaller the difference in two feature values, the more similar the two features are. But, due to the error in measurement, there is a deviation between the measurement value and the actual value. So, considering the measurement error in numerical feature similarity strategy is more reasonable, which could reduce the effect of measurement error when calculating similarity between two numerical features. According to this, we define (1) and (2)  In (1), ε is the measurement error threshold, and f (Di,k) e is the absolute value of numerical feature.
If f (Dj,k) e and f (Di,k) e are multi-dimensional numerical features, the dimensional similarity is calculated for the values in each dimension, and the feature similarity is the product of all dimensional similarities. If When calculating the feature similarity according to (1) and (2) k,t , it indicates that the difference between two numerical feature values (or two values in a certain dimension) is too large. In this case, we think that the two numerical features are not similar, the feature similarity value is set as 0.
(b) String feature similarity strategy Since each string represents a specific meaning, each string is considered as a whole to calculate the similarity. In Recognizer, according to the number of strings in string feature, the string features are divided into singlestring feature and multi-strings feature. We define (3) and (4) , we think that the feature value is incomplete. In this case, the feature similarity value is set to 0.8 (this is an experience value).
, we calculate the mean of intra-brand similarity increments according to (5).
According to ϕ f ( * ,m)  , we calculate ϕ f ( * ,m) e according to (5), and calculate the incremental standard deviation of intra-brand similarity according to formula (9).
According to ϕ f ( * ,m) (4) Recognize device type www.nature.com/scientificreports/ In device type recognition, there are two parts: brand recognition and model recognition. We first perform brand recognition on the target device, and then perform model recognition.
In brand recognition, brand features and normalized weights are used in (11) to calculate the similarity between target device and known devices.
In (11), T is the target device, K i is one known device in the knowledge set, and W(F b ) is the standardized weight vector of brand feature. In knowledge set, the brand of known device with the greatest similarity with target device is taken as the brand of target device. So as to realize brand recognition of the target device.
In model recognition, model features and normalized weights are used in (12) to calculate the similarity between target device and known devices. At this time, the brand of known devices is same with target device.
In (12), T is the target device, K i is one known device in the knowledge set (the brand of K i is same with target device), and W(F m ) is the standardized weight vector of model feature. The model of known device with the greatest similarity with target device is taken as the model of target device. So as to realize model recognition of the target device.

Analysis of Recognizer
In Recognizer, the brand features, model features and weights directly affect the accuracy of the recognition. We select brand features, model features and obtain their weights based on RFBR and RFMR strategies. Thus, in this section, we will analyze the rationality of RFBR and RFMR strategies.
Rationality analysis of RFBR strategy. RFBR strategy is used to quantify the importance of extracted features in brand recognition, and to select brand features and determine weights.
According to the research of Fu et al. 45 , the judgment criterion, that the selected feature is effective, is that the selected features can increase the difference between classes (we denote this criterion as effective feature selection criterion, abbreviated as EFS criterion). Therefore, in Recognizer, if one extracted feature could be selected as a brand feature, this extracted feature should be able to increase the difference between devices with different brands. In brand feature selection, there are two cases meeting EFS criterion: (1) For one extracted feature, if the feature can increase the similarity between devices with same brand and reduce the similarity between those devices with different brands, this extracted feature will be selected as brand feature. This is the optimal case. (2) For one extracted feature, if the feature can simultaneously increase the similarity between devices with same brand and similarity between those devices with different brands, and the ratio, between inter-brand-similarity increments and intra-brand-similarity increments, is less than threshold, this extracted feature will also be selected as brand feature. This is an acceptable case.
That means χ rqb (f ( * ,m) e ) > 0 is equivalent to that f ( * ,m) e satisfies the EFS criterion. The above analysis shows that RQB rule comply with EFS criteria, RFBR strategy can be used to evaluate the role of each feature in brand recognition, and it is reasonable to use RFBR strategy to select brand features. The value of features can be a significant reference for the selection of brand feature.

Rationality analysis of RFMR strategy. Because one model only corresponds to one mobile device,
there is no similarity between devices with same model. So, it is unreasonable to use RFBR strategy for model feature selection. For that, we use RFMR strategy to quantify the role of each feature in model recognition, and to select model features and determine weights. According to the EFS criterion, if one feature can be selected as model feature, the feature should be helpful to distinguish different models of device with same brand. This means that the similarity between two different model devices with same brand could be decreased after using this feature.
Assuming that f ( * ,1) Equation (19) shows that if f ( * ,m) e satisfies the EFS criterion, then (19) is correct. Similarly, if (19) is correct, f ( * ,m) e satisfies the EFS criterion. In Recognizer, we use RQM rule to obtain the value of χ rqm of extracted feature. When the value of χ rqm is greater than 0, the extracted feature will be selected as model feature. The larger the χ rqm value, the greater the feature weight. We can obtain (20) according to the RQM rule (10).
That means χ rqm (f ( * ,m) e ) > 0 is equivalent to that f ( * ,m) e satisfies the EFS criterion. At the same time, the incremental standard deviation of intra-brand similarity is another factor to assess the role of extracted feature in model recognition. According to (10), when the ϕ values of two brand features are same, the bigger the γ is, the smaller the χ rqm is. Since the standard deviation can be used to measure the degree of dispersion of data, the smaller the standard deviation, the more stable the data. Thus, when ϕ(f ( * ,i) if γ (f ( * ,i) The above analysis shows that RQM rule comply with EFS criteria, RFMR strategy can be used to evaluate the role of each feature in model recognition, and it is reasonable to use RFMR strategy to select model features. The value of features can be a significant reference for the selection of model feature.

Results and analysis of experiments
In this section, we first introduce our experimental dataset. On this dataset, we carry out three experiments: (1) experiment on selecting brand features and determining weights, (2) experiment on selecting model features and determining weights, (3) experiment on device recognition using Recognizer and other methods.
Experimental dataset. In this section, the experimental dataset includes 587 models of mobile phone devices from 17 brands. The brands and models of mobile devices are shown in Table 1.  Table 1 as extracted features: device dimensions, device weight, display size, screen-to-body ratio, resolution, display density, chipset, GPU, internal memory, memory card, operating system, battery capacity, battery charging, selfie camera, main camera, network technology, SIM, WLAN, NFC, Bluetooth, GPS. According to the form of extracted features, these features are numerical features: device dimensions, device weight, display size, screen-to-body ratio, resolution, display density, battery capacity, NFC, Bluetooth (where the device dimensions and resolution are multi-dimensional numerical features). And those extracted features are string features: chipset, GPU, internal memory, memory card, operating system, battery charging, selfie camera, main camera, network technology, SIM, WLAN, GPS (among which, battery charging, selfie camera, main camera, WLAN and GPS are multi-strings features). In our experiments, we set ε = 1 in calculating similarity between two numerical features.
Our dataset contains two parts: knowledge set and target set. The knowledge set and target set all include 587 different models of mobile devices, and the difference is: for any feature of mobile phone device, if there are multiple possible feature values, the target set only includes one possible value of the device. This means that the size of knowledge set is 587, but the size of target set is not less than 587. For example, the internal memory of iPhone 12 Pro has three possible values: "6 GB + 128 GB", "6 GB + 256 GB" and "6 GB + 512 GB".
Selecting brand features and determining weights. For each extracted feature, we calculate the average intra-brand similarity increment between two devices with same brand, and the inter-brand similarity (a) On each brand, intra-brand similarity increments of each extracted feature (b) On each brand, inter-brand similarity increments of each extracted feature www.nature.com/scientificreports/ increment between two devices with different brands. The effect of each extracted feature on the similarity of the devices with same brand and different brands is shown in Fig. 2.
As can be seen from Fig. 2, some extracted features, such as device dimensions, device weight, screen-to-body ratio, etc., can not only increase intra-brand similarity, but also increase inter-brand similarity. The reason may be that when manufacturers design mobile phones, they usually draw on the attributes of other brands, resulting in the similarity in some extracted features of phones with different brands. Therefore, in the acceptable case of EFS criterion, the extracted features, such as device dimensions, device weight and screen-to-body ratio, may be able to be selected as brand features.
We set α = 0.8 . According to (5), (6) and (8), we calculate the average intra-brand similarity increment ( ϕ ), the average inter-brand similarity increment ( δ ), value of χ rqb , and normalized weight ( ω b ). The results are shown in Table 3. Table 3 shows that those extracted features are selected as brand features: device dimensions, device weight, display size, screen-to-body ratio, resolution, display density, memory card, battery capacity, SIM, WLAN, NFC, Table 3. The values of ϕ , δ , χ rqb , and ω b of each extracted feature.  www.nature.com/scientificreports/ Bluetooth, and GPS. In device recognition experiment in this paper, those brand features will be used to recognize the brand of target device.

Selecting model features and determining weights.
For each extracted feature, we calculate the standard deviation of intra-brand similarity increment, and the result is shown in Fig. 3. For two extracted features, when average intra-brand similarity increments ( ϕ ) are same, the extracted feature with smaller standard deviation could stably decrease intra-brand similarity for all brand.
We set β = 0.8 . For each extracted feature, according to (9) and (10), we calculate the standard deviation of intra-brand similarity increment ( γ ), value of χ rqm , and normalized weight ( ω m ). The calculation results are shown in Table 4. The ϕ value of each extracted feature has been calculated in subsection Selecting brand features and determining weights, thus, we directly use the ϕ value in subsection Selecting brand features and determining weights when calculating χ rqm here. Table 4 shows that those extracted features are selected as model features: chipset, GPU, internal memory, operating system, battery charging, selfie camera, main camera. In device recognition experiment in this paper, those model features will be used to recognize the model of target device.
According to subsections Selecting brand features and determining weights and Selecting model features and determining weights, we build a feature set including 20 features. Those features are: device dimensions, device weight, display size, screen-to-body ratio, resolution, display density, chipset, GPU, internal memory, memory card, operating system, battery capacity, battery charging, selfie camera, main camera, SIM, WLAN, NFC, Bluetooth, GPS.  Table 5.

Device recognition using
In Table 5, "Brand Acc" is the brand recognition accuracy, and the value in parentheses below the accuracy value is the number of devices whose brand recognition results are correct in target set. "Model Acc" is the model recognition accuracy, and the value in parentheses below the accuracy value is the number of devices whose model recognition results are correct in target set. It is worth noting the model recognition result of device must be wrong, when brand recognition result is wrong. Table 5 shows that: (1) Recognizer, ProfilIoT, MSA and ByteIoT can recognize the brand and model of target device using our features. (2) the model recognition accuracy of Apple's mobile phone is significantly lower than other brands using Recognizer. So are the other three methods. We analyze model recognition results and find that Recognizer mistakenly recognized the phone model as other phone models in same series, such as recognizing "iPhone 11 Pro Max" as "iPhone 11 Pro", "iPhone 12 mini" as "iPhone 12", "iPhone XS Max" as "iPhone XS". We check feature values and found that values of all model features between misrecognized device model and true device model are same. That may be the internal reason of model misrecognition.
(3) For all mobile Table 4. The values of γ , χ rqm , and ω m of each extracted feature. Compared with the physical attributes, traffic features of device can be obtained easier. In our feature set, the resolution, operation system, and GPU of device can be obtained in the normal traffic. When only using the three traffic features, for different brands, the model recognition accuracy of Recognizer is shown in Table 6.
In Table 6, "Model Acc" is the model recognition accuracy. The results show that, when only using three traffic features, the model recognition accuracy of Recognizer is low. But we believe that more traffic features can improve the model recognition accuracy of Recognizer. Moreover, in some actual scenario, we can obtain some physical attributes of device (namely, all features in feature set may not be acquired simultaneously). Next, we use a part of features in the feature set to identify the brand and model of device.
We gradually reduce the number of used features from 19 to 2 (1 reduction each time). For each specific number of features (x 0 ), we randomly select × 0 features from 20 features of all devices in target set, and the other (20-x 0 ) features of all target devices are set null. In this way, we build 1 sample set (the size of sample set is equal to target set). For each x 0 , we build 1000 sample sets. Finally, Recognizer, ProfilIoT, MSA, and ByteIoT are used to recognize the brand and model of device in sample set. The relationship between recognition accuracy of the four methods and number of features is shown in Fig. 4. Figure 4 shows that: (1) brand recognition accuracy and model recognition accuracy of Recognizer, Pro-filIoT, MSA, and ByteIoT both decrease as the number of features decreases, and the recognition accuracy of Recognizer is greater than ProfilIoT, MSA, or ByteIoT. (2) When the number of features is less than 9, as the number of features decreases, both the brand recognition accuracy and model recognition accuracy of Recognizer  The above experimental results show that the model recognition accuracy of Recognizer is 99.08% (+ 9.25%↑) when using all 20 features in feature set. And when using any 13 features in feature set, the accuracy of Recognizer is 92.08% (+ 29.26%↑). The model recognition accuracy of Recognizer is higher than existing methods. This characteristic, that using a part of features in feature set also has a high recognition accuracy, is conducive to the widespread use of Recognizer.

Conclusion
In this paper, we propose Recognizer, a method to recognize the model of mobile device based on weighted feature similarity. We build a feature set including 20 features firstly. Then, we design RFBR and RFMR strategies to select features from feature set, and determine the weight of each feature. Finally, for target mobile device, based on all or part features in feature set, Recognizer identifies the model of target mobile device. The experimental results show that not only Recognizer, but also existing methods can use the features in feature set to recognize the model of mobile device. And the model recognition accuracy of Recognizer is greater than other four methods. when using all features in feature set, the accuracy of Recognizer is 99.08% (+ 9.25%↑). And when using any 13 features in feature set, the accuracy of Recognizer is 92.08% (+ 29.26%↑). In the process of recognition, some physical attributes are used in Recognizer, and a few of these physical attributes may be obtained by in-touch. Therefore, compared with existing traffic-based methods, the range of applications of Recognizer is limited. In future work, how to use fewer and easier-to-obtain features to recognize the model of device will be an important research direction.

Data availability
The information of mobile device is obtained on https:// www. gsmar ena. com/ search. php3.